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Abstract: The paper presents the navigation of mobile walking robot systems for movement in 
non-stationary and non-structured environments. In the first approach are presented main 
elements for the successful completion of intelligent navigation. The wireless sensor networks 
(WSN), dynamical stability control, strategies for dynamical control and a Bayesian approach of 
simultaneous localisation and mapping (SLAM) for avoiding obstacles and dynamical stability 
control for motion on rough terrain are studied. By processing inertial information of force, 
torque, tilting and wireless sensor networks (WSN) an intelligent high level algorithm is 
implementing using the virtual projection method. New capabilities to improve the walking robot 
stability are developed through the real-time balance motion control. The dynamic robot walking 
is presented in correlation with a stochastic model of assessing system probability of 
unidirectional or bidirectional transition states, applying the non-homogeneous/non-stationary 
Markov chains. The results show that the proposed new navigation strategy of the mobile robot 
using Bayesian approach walking robot control systems for going around obstacles has increased 
the robot’s mobility and stability in workspace. 
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1 Introduction 


Walking robots, unlike other types of robots such as those 
with wheels or tracks, use similar devices for moving on the 
field like human or animal feet. A desirable characteristic a 
mobile robot is the skills needed to recognise the landmarks 
and objects that surround it, and to be able to localise itself 
relative to its workspace. This knowledge is crucial for the 
successful completion of intelligent navigation tasks. But, 
for such interaction to take place, a model or description of 
the environment needs to be specified beforehand. If a 
global description or measurement of the elements present 
in the environment is available, the problem consists on the 
interpretation and matching of sensor readings to such 
previously stored object models. 

Moreover, if we know that the recognised objects are 
fixed and persist in the scene, defined as stationary 
environments, they can be regarded as landmarks, and can 
be used as reference points for self localisation. If on the 
other hand, a global description or measurement of the 
elements in the environment is not available, at least the 
descriptors and methods that will be used for the 
autonomous building of one are required (Looney, 1994). 
The approach of the localisation and navigation problems of 
a mobile robot which uses a WSN which comprises of a 
large number of distributed nodes with low-cost cameras as 


main sensor, have the main advantage of require no 
collaboration from the object being tracked. 
The main advantages of using WSN multi-camera 
localisation and tracking are: 
1 The exploit of the distributed sensing capabilities of the 
WSN. 


2 The benefit from the parallel computing capabilities of 
the distributed nodes. Even though each node have 
finite battery lifetime by cooperating with each other, 
they can perform tasks that are difficult to handle by 
traditional centralised sensing system. 


3 The employ of the communication infrastructure of the 
WSN to overcome multi-camera network issues. Also, 
camera-based WSN have easier deployment and higher 
re-configurability than traditional camera networks 
making them particularly interesting in applications 
such as security and search and rescue, where 
pre-existing infrastructure might be damaged 
(Jalilvand et al., 2009). 


Robots have to know where in the map they are in order to 
perform any task involving navigation. Probabilistic 
algorithms have proved very successful in many robotic 
environments. They calculate the probability of each 
possible position given some sensor readings and movement 


234 V. Viladareanu et al. 


data provided by the robot (Vladareanu et al., 2011; 
Kim et al., 2007). The localisation of a mobile robot is made 
using a particle filter that updates the belief of localisation 
which, and estimates the maximal posterior probability 
density for localisation. The causal and contextual relations 
of the sensing results and global localisation in a Bayesian 
network, and a sensor planning approach based on Bayesian 
network inference to solve the dynamic environment is 
presented. In the study is proposed a mobile robot sensor 
planning approach based on a top-down decision tree 
algorithm. Since the system has to compute the utility 
values of all possible sensor selections in every planning 
step, the planning process is very complex. 

The paper first presents the position force control and 
dynamic control using ZMP and inertial information with 
the aim of improving robot stability for movement in 
non-structured environments. This means moving the robot 
in sloping terrain, on steps or uneven environments which 
leads to modifications in the projection of the robot support 
surface and variable loads on the robot legs during 
movement. The next chapter presents the mobile walking 
robot control system architecture for movement in non- 
stationary environments by applying wireless sensor 
networks (WSN) methods. Finally, there are presented the 
results obtained in implementing the interface for sensor 
networks used to avoid obstacles and in improving the 
performance of dynamic stability control for motion on 
rough terrain, through a Bayesian approach of simultaneous 
localisation and mapping (SLAM). 


2 Dynamical stability control 


The research evidences that stable gaits can be achieved by 
employing simple control approaches which take advantage 
of the dynamics of compliant systems. This allows a 
decentralisation of the control system, through which a 
central command establishes the general movement 
trajectory and local control laws presented in the paper 
solve the motion stability problems, such as: damping 
control, ZMP compensation control, landing orientation 
control, gait timing control, walking pattern control, 
predictable motion control (Vladareanu et al., 2011, 2012). 
In order to carry out new capabilities for walking robots, 
such as walking down the slope, going by overcoming or 
avoiding obstacles, it is necessary to develop high-level 
intelligent algorithms, because the mechanism of walking 
robots stepping on a road with bumps is a complicated 
process to understand, being a repetitive process of tilting or 
unstable movements that can lead to the overthrow of the 
robot. The chosen method that adapts well to walking robots 
is the zero moment point (ZMP) method (Vladareanu et al., 
2010b; Vladareanu and Capitanu, 2012). A new strategy is 
developed for the dynamic control for walking robot 
stepping using ZMP and inertial information. This, includes 
pattern generation of compliant walking, real-time ZMP 
compensation in one phase — support phase, the leg joint 
damping control, stable stepping control and stepping 
position control based on angular velocity of the platform. 





In this way, the walking robot is able to adapt on uneven 
ground, through real-time control, without losing its 
stability during walking (Vladareanu et al., 2009b; Capitanu 
et al., 2008). 

Based on studies and analysis, the compliant control 
system architecture was completed with tracking functions 
for HFPC walking robots, which through the 
implementation of many control loops in different phase of 
the walking robot, led to the development of new 
technological capabilities, to adapt the robot walking on 
sloping land, with obstacles and bumps. In this sense, a new 
control algorithm has been studied and analysed for 
dynamic walking of robots based on sensory tools such as 
force/torque and inertial sensors (Vladareanu et al., 2010a; 


Raibert and Craig, 1981; Zhang and Paul, 1985). Distributed 
control system architecture was integrated into the HFPC 


architecture so that it can be controlled with high efficiency 
and high performance. 








3 Simultaneous localisation and mapping 


A precise position error compensation and low-cost relative 
localisation method is studied in Kim et al. (2007) for 
structured environments using magnetic landmarks and hall 
sensors. The proposed methodology can solve the problem 
of fine localisation as well as global localisation by tacking 
landmarks or by utilising various patterns of magnetic 
landmark arrangement. The research in localisation and 
tracking methods using WSN have been developed based on 
radio signal strength intensity (RSSI) and ultrasound time of 
flight (TOF). Localisation based on radio frequency 
identification (RFID) systems have been used in fields such 
as logistics and transportation but the constraints in terms of 
range between transmitter and reader limits its potential 
applications (Yoshikawa and Zheng, 1993). 

Many efforts have been devoted to the development of 
cooperative perception strategies exploiting the 
complementarities among distributed static cameras at 
ground locations, among cameras mounted on mobile 
robotic platforms, and among static cameras and cameras 
onboard mobile robots. 

Computation-based closed-loop controllers put most of 
the decision burden on the planning task. In hazardous and 
populated environments mobile robots utilise motion 
planning which relies on accurate, static models of the 
environments, and therefore they often fail their mission if 
humans or other unpredictable obstacles block their path. 
Autonomous mobile robots systems that can perceive their 
environments, react to unforeseen circumstances, and plan 
dynamically in order to achieve their mission have the 
objective of the motion planning and control problem 
(Stankovski et al., 2002; Vladareanu et al., 2010b; 
Deng et al., 2011; Fei et al., 2012). 

To find collision-free trajectories, in static or dynamic 
environments containing some obstacles, between a start 
and a goal configuration, the navigation of a mobile robot 
comprises localisation, motion control, motion planning and 
collision avoidance. Its task is also the online real-time 
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re-planning of trajectories in the case of obstacles blocking 
the pre-planned path or another unexpected event occurring. 
Inherent in any navigation scheme is the desire to 
reach a destination without getting lost or crashing into 
anything. The responsibility for making this decision is 
shared by the process that creates the knowledge 
representation and the process that constructs a plan of 
action based on this knowledge representation. The choice 
of which representation is used and what knowledge is 
stored helps to decide the division of this responsibility. 
Very complex reasoning may be required to condense all of 
the available information into this single measure (Shihab, 
2005; Iliescu et al., 2010). The techniques include 
computation-based closed-loop control, cost-based search 
strategies, finite state machines, and rule-based systems 
(Boscoianu et al., 2008; Rummel and Seyfarth, 2008). 
Computation-based closed-loop controllers put most of 
the decision burden on the planning task. In hazardous and 
populated environments mobile robots utilise motion 
planning which relies on accurate, static models of the 
environments, and therefore they often fail their mission if 
humans or other unpredictable obstacles block their path. 


Figure 1 Mobile robot control system architect 
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Autonomous mobile robots systems that can perceive their 
environments, react to unforeseen circumstances, and plan 
dynamically in order to achieve their mission have the 
objective of the motion planning and control problem. To 
find collision-free trajectories, in static or dynamic 
environments containing some obstacles, between a start 
and a goal configuration, the navigation of a mobile robot 
comprises localisation, motion control, motion planning and 
collision avoidance (Vladareanu_et_al., 2009b; Shihab, 
2005). A higher-level process, a task planner, specifies the 
destination and any constraints on the course, such as time. 
Most mobile robot algorithms abort, when they encounter 
situations that make the navigation difficult. Set simply, the 
navigation problem is to find a path from start (S) to goal 
(G) and traverse it without collision. The relationship 
between the subtasks mapping and modelling of the 
environment; path planning and selection; path traversal and 
collision avoidance into which the navigation problem is 
decomposed, is shown in Figure 1. 
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Motion planning of mobile walking robots in uncertain 
dynamic environments based on the behaviour dynamics of 
collision-avoidance is transformed into an optimisation 
problem. Applying constraints based on control of the 
behaviour dynamics, the decision-making space of this 
optimisation. 


4 Strategy for dynamical stability 


The walking robot is considered as a set of articulated rigid 
bodies, which are standing as a platform and leg elements. 
The static stability problem is solved by calculating the 
extremity of each leg position according to the system of 
axes attached to the platform, with origin at the centre of 
gravity of it. 

The proposed walking robot control strategy is 
based on three approaches, for conforming to movement 
characteristics: real-time balance control, walking pattern 
control and predictable motion control. The first main task, 
balance control, leads to a control model that periodically 
modifies the walking scheme, depending on the sensory 
information received from the robot transducers. 

In this paper we take into account the real-time balance 
control. 


Real-time balance control. The balance control, leads to a 
control model that periodically modifies the walking 
scheme, depending on the sensory information received 
from the robot transducers. Real-time balance control 
presented in Figure 2 contains four types of reactive loops: 
damping control, ZMP compensation control, walk timing 
control and walk orientation control. The second main task, 
walking scheme control, represents a real-time control of 
the robot equilibrium using the reactions of inertial sensors. 
The walking control scheme can be changed periodically in 
accordance with the information received from the inertial 
sensors during the walking cycle, by processing them into 


Figure 2 Real-time balance control of the walking robots motion 








two real-time loops: platform swing amplitude control and 
platform rotation/advance control. 

The third main task, predictable movement control, 
represents the control of predictable movement based on a 
fast decision from previous experimental data. Our research 
considers the following five dynamic control loops. Landing 
orientation control is achieved by integrating the torque 
measured for the entire gait and achieving a stable contact 
with the two ground surface by controlling the leg joint. A 
stable contact is obtained by adapting the leg articulations at 
ground surface, when an obstacle is preventing moving the 
leg on the required trajectory. The motion control will lead 
to a smooth walking. The control law for the landing 
orientation is: 

T(s) 


u, =u+———_ 1 
É Cis+K,; () 


where T is the measured torque, CL is the damping 
coefficient, KZ is the rigidity, u is the leg reference angle 
and u, is the leg’s joint compensated reference angle. 


Damping control aims to eliminate the oscillations that 
occur in the single support phase. The oscillations amplitude 
is measured in real-time by a torque transducer mounted on 
the robot joints, having compliant control functions of robot 
movement. A simple inverted pendulum equation with a 
joint in the single support phase, which opposes the 
damping forces of the leg joints was adopted for robot 
motion modelling. ZMP compensation control strategy 
consists in mathematical modelling of ZMP compensator 
through the spring-loaded inverted pendulum. A ZMP 
compensator is developed in single support phase (FSU), 
where the platform will move back and forth according to 
ZMP dynamics, because the damping loop is not sufficient 
to maintain a stable walking motion due to the ZMP 
movement influences. 
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Regarding the construction of mathematical model, based 
on quasi-dynamic analysis, each leg is considered as a 
generator function, with limited accuracy for displacement 
systems. If the number of degrees of mobility is equal to n 
and if the interior limitations have the following form: 





F, (xp x2, a: x, ) =0, j=l, m nèm (2) 


then, in the differential equations structure: 


de; 
dt 





= f, (x Ksera Xo Ujs Ugye Uae t), i=l, n (3) 


there are arbitrary u; coefficients, which are used to obtain 
the ‘stepping’ algorithm. For differential equations, the 
limitations imposed by the general (platform) base, where 
the legs are fixed, are applied in first case, and the 
limitations imposed on the supporting surface, secondly. 


5 Virtual projection method 


A virtual projection architecture system was designed which 
allows improvement and verification of the performance of 
dynamic force-position control of walking robots by 
integrating the multi-stage fuzzy method with acceleration 
solved in position-force control and dynamic control loops 
through the ZMP method for movement in non-structured 
environments and a Bayesian approach of SLAM for 
avoiding obstacles in non-stationary environments. By 
processing inertial information of force, torque, tilting and 
WSN an intelligent high level algorithm is implementing 
using the virtual projection method. 

The virtual projection method, presented in Figure 3, 
patented by the research team (Vladareanu et al., 2009a), 
tests the performance of dynamic position-force control by 
integrating dynamic control loops and a Bayesian interface 
for the sensor network. The CMC classical mechatronic 
control directly actions the MS1, MSm servomotors, where 
m is the number of the robot’s degrees of freedom. These 
signals are sent to a virtual control interface (VCI), which 
processes them and generates the necessary signals for 
graphical representation in 3D on a graphical terminal CGD. 
A number of n control interface functions ICF1-ICFn ensure 
the development of an open architecture control system by 
intergrating n control functions in addition to those supplied 
by the CMC mechatronic control system. With the help of 
these, new control methods can be implemented, such as: 
contour tracking functions, control schemes for tripod 
walking, centre of gravity control, orientation control 
through image processing and Bayesian interface for sensor 
networks. Priority control real-time control and information 
exchange management between the n interfaces is ensured 
by the multifunctional control interface MCI, interconnected 
through a high speed data bus. 


5.1 Bayesian Interface for sensor networks 


To determine the priors for the model parameters and to 
calculate likelihood function (joint probability) we define a 


given random variable x whose probability distribution 
depends on a set of parameters P = (Pı, P2, ... Pp). Exact 
values of the parameters are not known with certainty, 
Bayesian reasoning assigns a probability distribution 
of the various possible values of these parameters that 
are considered as random variables. Bayes’ theory is 
generally expressed through probabilistic statements as 
following: 


P(B|4) 


P(A| B) = P(A)x a 


(4) 


P(A | B) is the probability of A given the event B occurs or 
the posteriori probability. Using Bayes’ theory may be 
recurring, that if exist an a priori distribution (P(A) and a 
series of tests with experimental results Bı, Bo, ..., By..., 
expressed according to successive equations: 


P(B,| A) 
P(B) 
P(B, | A) P(B, | A) 
P(B) P(B) (5) 


P(A|B,)= P(A) 


P(A|B,,B,) = P(A) 





P(A|B,,By,...B, ) 
P(B, | A) 


= P(A|B,,By,...B,_,)- PUB) 


Figure 3 The virtual projection method 
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A posteriori distribution called also belief, is used when the 
test results are known, being obtained as a new function a 
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When there are no missing data or hidden variables the 
method for calculating P(B,;, D) for some belief-network 
structure Bs; and database D is presented in Looney (1994). 
Let Q be the set of all those belief-network structures that 
have a non-zero prior probability. We can derive the 
posterior probability of Bs; given D as: 


P(Bs;|D)= P(Bs,D)/D" Bs € OP(Bs;,D) (7) 


The ratio of the posterior probabilities of two belief-network 
structures can be calculated as a ratio for belief-network 
structures B,; and By, using the equivalence: 


P(Bs | D)/P(By | D) = P(Bs,D)/P(By,P) (8) 
which we can derive that: 
P(Bs,,D) = P(D|Bs;) P(Bs:) (9) 


Term P(B,;) represents prior probability that a process with 
belief-network structure Bs;. 

To designate the possible values of h, ca be used the 
Markov blanket method, MB(h) (Looney, 1994; Vladareanu 
et al., 2011). Suppose that among the m cases in D there are 
u unique instantiations of the variables in MB(h). Given 
these conditions it follows that: 


PDI- Di a MG) 
I, TT Pca |B,,B,) |f(BB, )aB, 


(10) 


Figure 4 The model with three states for the robotic system 


where G; is a given group contains c; case-specific hidden 
variables. 

Recall that u denotes only the number of unique 
instantiations actually realised in database D of the 
variables in the Markov blanket of hidden variable h. The 
number of such unique instantiations significantly 
influences the efficiency with which we can compute 
equation (10). For any finite belief network, the number of 
such unique instantiations reaches a maximum regardless of 
how many cases there are in the database. 

That r denotes the maximum number of possible values 
for any variable in the database. If u and r are bounded from 
above, then the time to solve equation (10) is bounded from 
above by a function that is polynomial in the number of 
variables n and the number of cases m. If u or r is large, 
however, the polynomial will be of high degree (Vladareanu 
et al., 2011). 

To model a robotic system requires considering 
in-between the two states of operating and faulting one or 
more intermediate states of partial success. In Figure 4 is 
considered a robotic system characterised by three states: 
operating at full capacity (F), defect (D) and intermediate 
(J). A generalised diagram of states is shown in Figure 5, 
which included three intermediate states. 
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The Markov modelling technique requires to identify each 
intermediate state (in practice, more neighbouring levels can 
be grouped together), to know the occupancy status of each 
component (7;) and the number of transitions between states 
(N;), which can calculate as follows: 


e occupancy probability of ‘7’ state: P = 


N, 


J 


T, 5) 





e transition intensity from state ‘7’ in ‘7: A; = 


where T} = be is analysed time interval. 


The number of intermediate states to be modelled in order to 
obtain a more accurate assessment of the reliability group is 
necessary to consider more than one intermediate state. 
Figure 6 presents a model with six states to assess the 
predictable transitions in a robotic system. The six states of 
the system are: 


pa 


operational state of robot 
2 landing control 

3 balance control 

4 advance control 

5 WSN control 

6 unpredicted event. 


Based on the surveillance data in operation regime of robot 
were determined transition probabilities using of the 


n.. 
relationship: p,; = —, where ny is the transition from state 
n; i 
‘i’ in ‘f’ in the analysis time interval; n; is the number of all 
transitions from state ‘i’ in any other states. 
Values of these transition probabilities are: 
Py, =0,217; P3 =0,29; Pig = 0,135; pis = 0,235; pi, = 0,123. 


By applying the method Markov chains are obtain the 


Figure 6 Modelling the states with possible transitions for robot 


occupancy probability of the sates for the robot: Pı = 0.27; 
P= 0.19; P; = 0.115; P4 = 0.235; Ps = 0.122; Ps = 0.068. 

The working diagram of the petri network is presented 
in Figure 7 (http://www-dssz.informatik.tu-cottbus.de). 

A token is assigned to P3, and is assumed that the 
localiser initially knows its position. The warning event ts 
fires when the localiser fails in estimating robot’s accurate 
position for several steps. Two navigation primitives can be 
modelled as Pı, P2, respectively. Initially, the robot selects 
its motion by a random switch comprising the transitions 
ti and ft, with corresponds to probabilities P? and P’., 
respectively. The transition between them takes place 
according to the change of localiser states. 

The immediate transition t; means that the robot takes 
Contour tracking as soon as the localiser Warning event 
fires. The other transition between two primitives, t and t4, 
are modelled as timed transitions in order to express that the 
robot can change its current navigation primitive during the 
localiser Success state, if necessary. 


6 Results and conclusions 


The control for walking robots is achieved by a control 
system with three levels. The first level is to produce control 
signals for motor drive mounted on leg joints, ensuring the 
robot moving in the direction required with a given speed. 
The language for this level is that of differential equations. 

The second level controls the walking, respectively it 
coordinates the movements, provides the data necessary to 
achieve progress. At this level, work is described in the 
language of algorithms types of walking. 

The third level of command defines the type of walking, 
speed and orientation. At this level, the command may be 
provided by an operator who can use the control panel, in 
pursuit of its link with the robot, to specify the type of 
running and passing special orders (for the definition of the 
vector speed of movement). 
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Figure 7 The petri network diagram (see online version for colours) 
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To maintain the platform in a horizontal position, the 
information provided by the horizontality transducers 
(or verticality) is used, that sense walking robots deviation 
platform to the horizontal position. Restoring the horizontal 
position of the platform is achieved at the expense of 
vertical movement of different legs of support, as decided 
by the block to maintain balance. Returning to the fixed 
height of the platform is achieved by using information 
provided by the height transducer of the platform and by 
simultaneous control of vertical movement of all legs in 
support phase. 

From the analysis performed results the effectiveness of 
the proposed control strategy for a walking robot. The 
position of each actuator is controlled by a PD feedback 
loop, using encoder like transducers. 

In HFPC control system, the PC system sends 
the reference positions to all actuators controllers 
simultaneously at an interval of 10 ms (100 Hz). Reference 
positions for the control of 18 actuators and actual positions 
on each axis robot obtained through interpolation are 
processed at an interval of 1 ms (1 kHz). 

The ready to walk position is a robot base position, 
before the actual walk. For this position, the robot lowers 
the platform by bending the leg joints. The reason is to 





prevent the singular problem of the inverse cinematic and to 
achieve a stable walking with a constant platform height 
from the ground. To be observed that the platform height is 
linked to the dynamic properties of the robot. When the 
robot walks, it is periodically in the unique support phase. In 
this phase, the robot can be similar to the simple inverted 
pendulum model on the coronal plane and its natural 
frequency is: 


-! [Soy 
fot fu (11) 


where g and / are the desired acceleration due to gravity and 
respective the height from the ground of the robot’s centre 
of mass. 

Certainly, the natural frequency of the simple inverted 
pendulum exists in theory, because the robot’s tilt is limited 
by a specific angle. Thus, one can determine the walking 
period for a smooth motion in two phases (for the tripod 
walking) and efficient power consumption. For example, for 
a robot with the height | of approximately 900 mm and the 
balance of 40 mm results the natural frequency of 0.526 Hz. 
Figure 8 shows the general configuration of the HFPC 
system for ZMP control method. 
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Figure 8 Open architecture system of the walking robot 
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The control system is distributive with multi-processor 
devices for joint control, data reception from transducers 
mounted on the robot, peripheral devices connected through 
a wireless LAN for offline communications and CAN fast 
communication network for real-time control. The HFPC 
system was designed in a distributed and decentralised 
structure to enable development of new applications easily 
and to add new modules for new hardware or software 
control functions. 

The proposed petri nets and Markov chains approach 
provides a promising solution towards the development 
quantitative approach of dynamic discreet/stochastic event 
systems of task planning of mobile robots. For a deeper 
insight into control and communication of governing task 
assignment of the robot, the entire discrete-event dynamic 
evolution of task sequential process have to be linguistically 
described in terms of representations. 

This approach has the potential to model more complex 
relationships between target parameters. Moreover, the short 
time execution will ensure a faster feedback, allowing other 
programs to be performed in real-time as well, like the 
apprehension force control, objects recognition, making it 
possible that the control system have a human flexible and 
friendly interface. 
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